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Abstract 

Taking a phenomenological approach, we study a color sextet scalar at the LHC. We focus on 
the QCD production of a color sextet pair <I>6^6 through gg fusion and qq annihilation. Its unique 
coupling to V'^V' allows the color sextet scalar to decay into same-sign diquark states, such as 
$6 ~^ tt/tt*. We propose a new reconstruction in the multijet plus same sign dilepton with missing 
transverse energy samples {bb + i^i^ +0t + Nj, > 6) to search for on-shell tttt final states from 
sextet scalar pair production. Thanks to the large QCD production, the search covers the sextet 
mass range up to 1 TeV for 100 fb~^ integrated luminosity. 
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I. INTRODUCTION 



The Large Hadron Collider (LHC) at CERN will soon provide a great opportunity for 
exploring physics at the TeV scale. As a proton-proton collider with a total center-of- 
mass energy of 14 TeV, the LHC is truly a Quantum Chromo dynamics (QCD) machine. 
We therefore wish to study color exotics, since any accessible new physics in the strong 
interaction sector will appear in the early stages of LHC operation. Many models of physics 
beyond the standard model (SM) naturally require the presence of color exotics, such as 
gluinos and squarks in supersymmetric extensions of standard model, i^'i^'-gluons and KK- 
quarks in extra dimensional models, or the top-prime in Little Higgs or twin Higgs models. 
All of these are either quark or gluon partners which belong to the fundamental and adjoint 
representations of the QCD gauge group SU{?>)c respectively. It is natural to consider 
colored particles in other representations; in this paper, we focus on a scalar which is in 
the sextet(6) representation of SU{?))c- Color sextet particles have been widely discussed in 
nuclear physics as diquark condensate states; in the SSC era, sextet quarks were discussed 
in Q. 

Color sextet scalars are naturally present in partial unification Q|, grand unification 
and composite models; in some cases they may be present around the weak scale. For 
instance, in a supersymmetric Pati-Salam SU{2)r x SU{2)l x S'f/(4)c model, light color 
sextet scalars can be realized around the weak scale, even though the scale of SU(2)pi x 
SU{A)c symmetry breaking is around 10^° GeV due to existence of accidental symmetries 
with the masses of color sextet Higgs arising only through high dimension operators 
In this case, the introduction of a color sextet Higgs will not lead to proton decay but only 
to neutron-anti-neutron (n — n) oscillation and is fully compatible with present limits 0, Isj. 
^n a similar framework, light color sextet scalars also help in Post-Sphaleron baryogenesis 
3]. In this paper, however, we will take a purely phenomenological approach toward the 
sextet scalar without assuming any model a priori. 

Among all the color exotics, the color sextet scalar is unique in its coupling to quarks. 
In group theory language, the color sextet lies in3®3 = 6©3asa symmetric 2"^^^ rank 
tensor under SU{3)c- The Lorentz structure for this scalar coupling to quarks is given by 
tlj'^C~^ip(f), where ip is a Dirac spinor and (p is the scalar. Under the SM gauge group SU {3)c x 
SU{2)l X f/(l)y, the sextet scalar can be Ae, a SU{2)l adjoint (6,3,1/3); $6, a SU{2)l 
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singlet (6,1,4/3); 06, a SU{2)l singlet (6,1,-2/3); or Sq, a SU{2)l singlet (6,1, +1/3). 
The color sextet scalars are also charged under the global baryon symmetry U{1)b and the 
electromagnetic symmetry U{1)em- To avoid breaking U{1)em, these scalar fields should 
not develop a non-zero vacuum expectation value. This condition removes any possibility of 
n — n oscillation in the minimal model involving color sextet scalars. We may write down 
the fiavor independent Lagrangian of such a minimal model by only considering SM gauge 
invariants and keeping f/(l)EM unbroken, 

C = Tr[(D^A6)t(Z}^A6)]-MiTr[A^A6] + /AgIC-V2A^gi 
+ (D^$6)t(/^''$6) - Ml^l^, + UulC'uR^l 
+ (D^06)^(^^06) - Ml^eVe + UdlC~'dR<pl 

- AA(Tr[A^A6])^ - A^($^$6)^ - X^^^h? - XsiSe^S,)' 

- X'^Tt[AIA,AIA,] - Tr[AjA6](Ai$6t<f6 + Aa^eVe + X,6,^6,) 

- A4$6^$606V6 - A5$6"^$656"^56 " Ae^eVef^G^^e , (1) 

where the QCD covariant derivative is defined as = — igsC^^T^ , and the are 
the representation matrices for the sextet; Mf, Aj and fi are all positive-definite model 
parameters. 

If we consider the SU{2)l adjoint sextet scalar Ag , there will be three physical sextet 
scalar states that couple to up-type quark pairs, down-type quark pairs, and up-down type 
quark pairs. When the sextet scalar decays into light quark states, the existing search 
strategies for massive octet scalars or vectors 8| may be employed. Ag, $6 and 5^ may all 
contribute to the single top plus jet singal and ti + Nj signal from pair production. Here 
we consider the scenario in which a color sextet scalar decays into a top quark pair so that 
one can use the leptons from the top quark decay to determine the features of the sextet. 
Signature that contains multi-top final states has been discussed in the context of many new 
physics models as resonance decaying into top quarks or top composite [5|, 16|. To illustrate 
this and simplify our search, our study will focus on the color sextet SU{2) singlet scalar $6 
that only couples to righthanded up-type quarks. 
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II. DECAY OF THE COLOR SEXTET SCALAR 



The decay of the $6 depends on its mass, M$g, and its couphngs to quarks, /«-,• = 
u, c, t). To illustrate our reconstruction algorithm in the discussion of discovery, we consider 
the case where M$g > 350 GeV and the $6 decays into two onshell top quarks; other mass 
ranges are discussed in the conclusion section. Above threshold, the general expression for 
the decay partial widths of the sextet scalar are 

iOTT 

r., = ^\h\'M^^'^\l,rlr^){l-r^-r]) (2) 

where X{x, y, z) = {x — y — — Ayz and rj = nii/M^^. 

By far, the most stringent bounds on these parameters come from D° - D° mixing, to 
which $6 would make a tree level contribution proportional to fnf22/M^^. The off-diagonal 
coupling fij will contribute to flavor violation processes, for instance D ^ mi which is 
proportional to fufii/M^^. The current bounds require that 

/ii/22<10-^;/ii/i2< 10-^ (3) 

for M$g of a few hundred GeV to TeV mass range . One will also expect less stringent 

constraints from one loop process as c ^ M7. To escape from the bound, for acessible values 
of M$g we expect at least one of the couplings, /u or /22, to be negligible. However from our 
purely phenomenological perspective, we take the decay branching fraction BR($6 ~^ tt) to 
be a completely free parameter whose value may be determined at the LHC 

Because the sextet is a colored object, we need to consider the possibility of it hadronizing 
before decaying. For example, it may form a tetraquark-like bound state with 33, such 
as $6^^; $6^*^; or ^Qcld, with charges 0,1,2 respectively. If the total width is less than 
Aqcd ~ f^n, then the colored object will hadronize before it decays. To determine the 
constraint imposed by the possibility of hadronization, in Fig. [T] we plot the contour for 
which decay width of $6 is equal to Aqcd as a function of the couplings and the mass. 
Setting fuu = 0.001, fut = 0.001, and eliminating any coupling to c, we see the possibility 
that a large portion of our parameter space will be protected from the risk of hadronization. 



4 




FIG. 1: Decay width contour for $e in the mass and couphng plane. 
III. PRODUCTION OF $6 

Because it carries color, $g can be produced directly through the QCD strong interaction 
at the LHC The pair of $6*^6 is produced from gluon-gluon fusion or qq annihilation: 

9{Pi) + 9{P2) ^(i{ki) + $6(^:2) 

q{pi) + q{p2) $6(A;i) + ^%{k2). (4) 

The total production cross section depends only on the mass of $6? since the vertex is just 
the strong coupling, Qg, as shown in Eq. ([6]). By comparison, the electroweak production of 
$6 $6 is small enough to be neglected in our search. 
From the scalar QCD gauge interaction 

(Z}^<I>6)^P'^$6), where = - igsG^T^ (5) 
one may obtain the Feynman rules 

GlGl^,^, : -iglg,,{T-T' + T'T'^). (6) 

The momenta are assigned according to V^>S'(pi)S'(p2) and all momenta are out-going. In 
group theory langauge, this is6(8)6 = 27©8©l. 
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dR 


3 6 8 


C{R) 


1/2 5/2 3 


C2{R) 


4/3 10/3 3 



TABLE I: Normalization factor C{R) and quadratic Casimir C2{R) for dn = 3,6,8 under SU{3). 



The parton level cross sections for color a sextet pair production are given by 

, ds n3 _ 2 



and 



1 

a{gg -> ^g^g) = rf«C2(i?)7r^^[3/?(3 - ^f) - I2C2 - 2) 
+ ln|^|(6C2(i?)(/?^ - 1) - 9(/?2 - \f\ 



^TT /? -I- 1 

^a^[/3(89 - 55/3^) + ln|^|(ll/3^ + \%(? - 29)] 



(7) 



(8) 



where ^/s is the total energy, /3 = y 1 — 4M|g/s and i? is 6 with the normalization factor 
C and Casimir C2 satisfying 



Tr [T^T^] = and T%Tl = C2{R)1. 



(9) 



We list the values for different representations under SU (3) as in Table [B 

The QCD production cross sections for the color sextet scalar pair i'e'^e at both LHC 
and Tevatron are plotted in Fig. [2] with factorization scale = M$g , renormalization scale 
/iij = mz and the CTEQ6L 11| parton distribution function (PDF). The matrix elements 
in our calculations here and elsewhere are generated by SUSY-Madgraph with modified 
color factors. For comparison, we also show the pair production cross sections for SU{3)c 
triplet and octet scalars at the LHC. As we can see, the total production cross section of 
the sextet scalar is similar to that of the octet scalar, but is about one order maginitude 
larger than that of the triplet scalar, which can be understood from values of C and C2 for 
different reprensentations in Table [B 

As discussed in the introduction, the color sextet scalar $6 only couples to the righthanded 
up-type quark quark pair. Thus we may also have single production of a $6 through 



uu{cc) — > $6- 



(10) 
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FIG. 2: Production of ^q^q at the LHC and Tevatron with = M$g, fixed scale asifJ-u) with 
fJ-R = rnz- The PDF set CTEQ6L has been used in all calculation. 

However, the production cross section is proportional to the couphng \fuu\^ and |/ccP, and 
may therefore be suppressed due to the — Z)° mixing constraint. Some studies of the 
single $6 production at the Tevatron and the LHC have been done in Ref. j^. 



IV. SEARCHING FOR THE COLOR SEXTET SCALAR THROUGH tttt 

As discussed in the previous sections, the most distinct feature of the color sextet scalar 
is its decay mode $6 ~^ tt, which leads to a same-sign dilepton signature in the final state 
if both top quarks decay semileptonically, i.e. t W~^b — > i'^ub. To avoid ambiguities in 
lepton assignments during reconstruction, we require the anti-top quark pair from the $6 to 
decay hadronically. Hence, the final state of $6$6 is 

pp i>6$6 -> tttt ^Ab + +Xt + iVj, (11) 

where £ = e and yU and > 4 to allow initial and final state QCD radiation. In our study, 
however, the QCD radiation is not included. To get this final state, the decay branching 
ratio will be 22 

BR = BR2($6 ^ tt) X X X 2, (12) 
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where the situation that top quark decays hadronically and anti-top quark decays semilep- 
tonically is also included. Figure [2] also clearly shows that a color sextet with M$g > 350 
GeV will not be bounded by Tevatron data as the same-sign dilepton plus multi-jet final 
state from tttt will be less than one event for 2 fb~^ luminosity. 

To illustrate the kinematic features of the color sextet scalar pair, we consider the decay 
process $6*6 tut bbbbi+i + + ijets and take = 600 GeV. The leading and 

second-leading jet pt distributions are shown in Fig. [31 The typical hardness of these jets 
is the basis for one of our selection cuts introduced later in this section. In order simulate 
the detector effects on the energy-momentum measurements, we smear the electromagnetic 
energy and the muon momentum by a Gaussian distribution whose width is parameterized 



as 



ApT 



'cah Ctcal = 5%, bcal = 0.55%, (13) 
dtrack = 15%, btrack = 0.5%. (14) 



Pt TeV Vsin^ 

The jet energies are also smeared using the same Gaussian formula as in Eq. f|T3l) . but with 

Q 

a^i = 100%, b,ai = 5%. (15) 



We first reconstruct the two on-shell hadronically decaying W^s. Our procedure is to 
consider all dijet invariant masses except for those containing one of the two tagged b-jets, 
since we require b-tagging in the event selection discussed later, and choose the two closest 
Mjj combinations, which we then require to lie within the mass window 

\Mjj-mw\<l5GeV. (16) 

From this, we get the two reconstructed W momenta. We then consider all combinations of 
reconstructed pw with all jets and again choose the two closest invariant masses Mjw- In 
this way, we reconstruct the two hadronically decaying anti-top quarks. The distributions 
of these reconstructed invariant masses are shown in Fig. HI 

Once we have the reconstructed two anti-top quarks, the reconstruction of the sextet 
($) can be done using the 6-jet invariant mass Mgj for the two hadronic anti-top quarks. 
Although the production of neutrinos prevents us from fully reconstructing the sextet which 
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FIG. 4: Reconstructed Hadronic Top Pair. The black(red) line represents the first(second) recon- 
structed hadronically decaying anti-top quark. 

produces the leptonic decays, we may reconstruct the transverse mass Mt for the remaining 
two jets plus same-sign dilepton and as 



(17) 
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FIG. 5: Reconstructed Sextet from mgj and Mt- 
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FIG. 6: Distribution of cos 9 between reconstructed top momentum and reconstructed sextet mo- 
mentum. Dashed(Solid) line shows the distribution without(with) smearing effects and kinematic 
cuts. 

As seen in Fig. [5], our reconstruction shows a clear resonance in both the Mgj and Mt 
distributions. 

Finally, since the two anti-top quarks may be fully reconstructed, we can boost back to 
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the rest frame of the $6 and study its spin. As shown in Fig. [6l the angular distribution of 
the anti-quark clearly shows that the $6 is a scalar. Since there are two missing neutrinos 
from decay of two top quarks, it is challenging to fully reconstruct top quark's momentum, 
and study the spin information of top quarks, which can be used to check this model since 
$6 only decays into a right-handed top quark pair. We leave this for future work. 

We next consider the backgrounds for our signal. We require at least 2 tagged b-jets 
plus a same-sign dilepton and multijet. The irreducible SM background for this final state 
consists of ttW"^ + Nj, bb + VV^W^ + Nj and ttti. We estimate the QCD bb + W^W"^ + Nj 
background by computing jjW^W^ production. This is only 14 fb, and one expects the 
bb + W^W^ + Nj is about three orders lower and therefore < 0.1 fb. The SM 4-top tttt 
is less than 0.1 fb to start with. The leading background thus comes from ttW^ with one 
hadronic top decay and one semileptonic top decay with the same sign as leptonic decay. 

We propose the following selection cuts: 

• mm{pT{j)} > 15 GeV, max{pr(j)} > 100 GeV, next-to-max{pT(j)} > 75 GeV, 
IvU) < 3.0| 

• same sign dilepton with pt(^) > 15 GeV, \ri{i) < 2.8 1 

• ARjj, ARji, ARu > 0.4 

• at least two b-tagged jets 

• Xt > 25 GeV 

Since the production rate of our signal only depends on the mass M$g and branching 
ratio of $6 decay to a top quark pair, we scan these two parameters to study the discovery 
potential. We summarize our results in Fig. [7] as the signal production rate for bbbb + 
£^i^ + 0T + 4j from $6^6 with SM tiW^ background included. We use a factor of 25% 
in both plots in Fig. [7] for tagging two b jets with 50% effeciency to tag each 6-jet. The 
SM background is taken as 1 fb in the significance contour. As we can see in the left plot 
of Fig. [71 for 100 fb~^ luminosity, the statistical significance can surpass the 5a level for 
M$g < SOOGeV if BR($ tt) is about 0.5. Also note that the mass of the sextet scalar can 
be determined by reconstructing two hadronically decaying top (or anti-top) quarks, and 
the branching ratio of $6 ^ can be roughly estimated from the total signal event rate 
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FIG. 7: Production Rate normalized by BR(<56 ~^ tt)^ and Significance Contour. Dashed(solid) 
curves in left plot represent production before(after) selection cuts. 

if the one can understand the backgorund sufficiently well. No reconstruction selection has 
been implented since we did not simulate the events with initial state/final state radiation 
and the reconstruction efficiency is thus unknown. In principle, we expect that the S/ \/B 
can be further improved by including reconstruction. 

V. CONCLUSION 

In this paper, we discuss the production of a new exotic particle, a color sextet scalar, 
at the CERN Large Hadron Collider. Taking a purely phemenological approach, we discuss 
the discovery of the color sextet scalar through its decay into a top-top quark pair. The 
unique feature of same sign dilepton plus multijet makes it easy to identify and reconstruct 
the color sextet scalar object. Due to its large QCD production, it is possible to cover the 
color sextet scalar up to a mass range of 1 TeV for 100 fb~^ integrated luminosity. 

In the text, we only consider the case of M<i,g > 2mj, where $6 decays into two on-shell 
top quarks. In the case 2M$g < mz, there is a possiblity of a Z decaying into a sextet pair, 
since $6 carries a U{1)y charge, which we expect is highly constrained by LEP data. We 
also expect to find strong constraints from Tevatron data. For example, for M$g just above 
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rrit + rrib threshold, the $6^6 signal will directly contribute to ttX sample as the offshell top 
decay products are soft. 
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